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This study is intended to examine the influence of the democratic climate of 
classrooms on student engagement and learning outcomes in order to find out a more 
adequate model of learning in Civic Education classrooms. A model is developed for 
testing with data obtained from a sample of 930 students from schools in North 
Sulawesi. Prior to the analysis, sccdes are analysed using Confirmatory Factor 
Analysis (CFA) and are ccdibrcited using Rasch Measurement Model Analysis 
(RMMA). The analysis shows that the trimmed model (Model B) is slightly more 
coherent and simpler than the hypothesised model (Model A). However, both models 
indicate that the democratic climate of Civic Education classrooms has significant 
effects on student engagement, student civic knowledge and interpretation skill, and 
student concepts of citizenship. 

Civic education, democratic climate, classrooms, student engagement, citizenship 


Johnson and McClure (2004) define the classroom learning environment as a social atmosphere in 
which learning takes place that is sometimes called the educational environment. Kubow and 
Kinney (2000) argue that educational environment is related to how teaching is conducted in the 
classroom setting. More specifically, Moos (1979) conceptualises educational environment as a 
system that have four variables, namely physical environment, organisational aspects, teacher 
characteristics and pupil characteristics in which classroom climate is viewed as the mediator 
between these variables that operated through interactions among class members, teachers and 
students. This process is influenced by the orientation, the quality and the quantity of interactions 
and intercommunications between the classroom members (Allodi, 2002). These, in turn, affect 
student satisfaction, self-concept and the learning processes that influence learning outcomes. 

Research into classroom environments has been carried out over many years. Different studies 
have been undertaken to investigate a variety of aspects of classroom environment. These studies 
have ranged from investigating factors influencing learning environments to the students’ 
perceptions of their classrooms, and the relationship between student perceptions of their 
classrooms and their learning outcomes. These studies have been extended to cover schools and 
families (Parsons, 2002). In addition, the researchers have conducted studies about the effects of 
the classroom environment on the learning of different subject matter in different parts of the 
world. Aikin (1942) studied the effects of democratic processes in the classrooms and the schools 
in the Eight Year Study in the United States (reported in Morgenstem & Keeves, 1997). Kim, 
Fisher and Fraser (1999) investigated science classroom environments; Waldrip and Fisher (2003) 
investigated the differences between urban and country student perceptions of their learning 
environment (reported in Dorman, 2003); and Guthrie and Cox (2001) investigated the school and 
classroom context that would make students want to engage in reading longer. 


1 Preparation of this paper was supported by the Cultural Inclusivity through Publishing Project and funded by a 
Flinders University Diversity Initiative Grant. 
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Researchers in this field have divided engagement in classroom learning into three categories, 
namely behavioural, cognitive, and emotional engagement. Behavioural engagement consists of 
actions like following the rules, adhering to the classroom norms, and the absence of disruptive 
behaviours such as skipping school or getting into trouble, participating in classroom learning and 
academic tasks, persistence, effort, attention, asking questions, and participating in school-related 
activities. Emotional engagement includes student positive and negative affective reactions in the 
classroom, students’ emotional reactions to the school and the teacher, feeling of being important 
to the school, and valuing success in school-related outcomes. Cognitive engagement is 
conceptualised in terms of a psychological investment in learning, a desire to go beyond the 
requirements of school, and a preference for challenge by being strategic or self-regulating 
(Fredricks, et al., 2003, 2004). 

Studies have been undertaken to identify the correlation between behavioural engagement and 
learning outcomes for elementary and high school students (e.g. Alvermann et al., 1987; Ames, 
1992; Finn et al., 1995; Guthrie & Cox, 2001; Miller & Meece, 1999). Other studies have focused 
on the correlation between discipline problems, behavioural disengagement and achievement 
across grade levels (e.g. Aikins et al., 2005; Barker & Gump, 1964; Bates et al., 2003; Finn & 
Pannozzo, 2004; Fredricks & Eccles, 2002). The finding is that behavioural engagement has long- 
term effects on student performance. Students who show engagement and interest in their early 
grade levels are found to be performing better in their later years (Fredricks et al., 2003). 

Brown (1997), and Turner and Scott (1995) emphasise that social discourse in learning 
communities is intrinsically motivating. Furthermore, Wentzel (1991, 1997, 2002, 2003), Urdan 
and Maehr (1995) demonstrate that student possession of pro-social goals lead to their 
constructive social behaviours in the classroom. Fredricks, Blumenfeld, Field and Paris (2002) 
found that there was a unique relation of a challenging and a structured work environment 
involving student affect, behaviour and cognition. Skinner and Belmont (1993) also found that 
there was a reciprocal relationship between teacher behaviour and student engagement in the 
classroom. Teachers’ interactions with students predicted student behavioural and emotional 
engagement in the classroom, both directly and indirectly through their effects on student 
perceptions of their interactions with teachers. In addition, Kindermann (1993) and Wentzel 
(2002) argue that there is an association between children peer groups and the amount of 
engagement the children’s showed in the classroom. Palincsar (1998) in the analysis of the theory 
of constructivism claims, that the growing interest in social constructivist perspectives is 
propelled by recent educational reform efforts encouraging students to assume a more active role 
in their learning, to explain their ideas to one another, to discuss disagreements, and to cooperate 
in the solution of complex problems, while teachers participate in the design of these contexts and 
the facilitation of this kind of activity. However, to make an effective context for learning, 
discourse must be communicative. These findings confirm the importance of learning 
environment in fostering student learning (Clark et al., 2003; Belenky, 1997; Greeno 1998; 
Randolph, 2000; Roeser, Midgley & Urdan, 1996; Ryan & Partick, 2001; Turner et al., 1998; 
Wentzel, 2002). 

Torney-Purta et al. in their cross-national studies on Civic Education reports that a democratic 
climate of classrooms has a positive effect on student civic knowledge (1975; 2001). 

Based on these research studies, it is argued that it is important to encourage students engagement 
to learning activities in democratic Civic Education classrooms in order to provide them with 
opportunities to obtain deeper understanding of the civic values transmitted through meaningful 
classroom experiences in order to enable them to implement their civic values critically and 
responsibly in their social interactions. 



Mapiasse 


395 


METHOD 

Instruments 

In order to measure the constructs of the model of this study, instruments were developed by 
operationalising the concepts into items with categories of meaning that were measurable. 

Scale for Democratic Climate of Civic Education Classroom (DCCEC) was constructed basically 
based on a concept developed by Kubow and Kinney (2000) augmented by concepts suggested by 
Radz (1983) and Bickmore (1993). Scale for Student Engagement in Civic Education Classroom 
(SECEC) was constructed by adapting and modifying a scale previously developed by Fredricks 
et al. (2002, 2003, 2004). Scales for Civic Knowledge and Interpretation Skill (CKIS), Student 
Concepts of Democracy (SCD) and Student Concepts of Citizenship (SCC) were developed by 
adapting scales previously used by Torney-Purta, et al. (1975, 2001) in IEA study, MCEETYA 
(2004) and Mellor (2004), and modifying them based on Indonesian Civic Education curriculum 
and text books (i.e. DIKNAS, 2003, 2004; Dwiyono et al., 2003). The number of items prepared 
for these scales were 33, 19, 58, 24 and 27 respectively. A sample is provided in Table 1. All 
items were delivered to students in Bahasa Indonesia. 


Table 1. Sample of Scales and Items 


Scale 

Dimensions 

Descriptions 

Item samples 

Democratic Climate of Civic Education Classroom (DCCEC) 

Democratic 
climate of 
Civic 

Education 

Classroom 

Active 

participation 

Sharing ideas and among peers and facilitators; 
teachers do not take a stand as an authority 
on civic subjects; teachers share impressions 
on class activities with students. 

We feel free to share ideas in class. 

Student Engagement in Civic Education Classroom (SECEC) 

Student 

engagement 

in 

Civic 

Education 

Classroom 

Behavioural 

engagement 

Positive conducts (adhering to classroom 
norms), absence of disruptive behaviour such 
as skipping classrooms and getting in trouble; 
involvement in learning such as effort, persistence, 
concentration, attention, asking questions; 
and contribution to class discussions, participation 
in classroom activities. 

I pay attention in die Citizenship 
Education class. 

Student Civic Knowledge & Interpretation Skill (SCKIS) 

Civic 

knowledge 

Key features 
of Indonesian 
democracy. 

Understand that Indonesian citizens use secret 
ballots to elect representatives to govern on their 
behalf. 

In Indonesian democracy, how do 
citizens elect representatives for 
the People's Representative Councils 

Student Concepts of Democracy (SCD) 

Concepts 

of 

Democracy 

Freedom 

Students' beliefs about what is good or bad for 
democracy with respect to freedom. 

When everyone has die right to 
express then opmions freely that is . . . 

Student Concepts of Citizenship (SCC) 

Concepts 

of 

citizenship 

Effective 
participation 
in democratic 
decision- 
making 

A belief that in a democratic society people are 
entitled to hold and express dieir views on civic 
and political matters, within the law, and in turn 
must respect die others to do die same. 

An adult who is a good citizen 
would participate in a peaceful 
protest against a law believed to be 
unjust. 

As shown in Table 2, a four point Likert scale using a 0, 1,2 and 3 scoring scheme was used for 
Democratic Climate of Civic Education Classroom (DCCEC), Student Engagement in Civic 
Education Classroom (SECEC), Student Concept of Democracy (SCD) and Student Concepts of 


Citizenship (SCC) scales. For the Civic Knowledge and Interpretation Skill (CKIS) scale, a four- 
alternative multiple choice scheme was used. 

Complexity of the sample structure entailed the two stages of sample selection, namely, the 
school level stage using the Probability Proportionate to the Size (PPS) procedure and the student 
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level stage using a Simple Random Sampling (SRS) procedure (Rosier & Ross, 1992). PPS and 
SRS procedures gave students the same opportunity to be in the sample at the school level and 
student level. In selecting students from each school, an intact class was used because the main 
focus of the study was a classroom environment. 


Table 2. Category meanings 


Constructs 


Category meanings 



0 

1 

3 

Democratic Climate of Civic 
Education Classroom & 
Student Engagement 

never 

rarely sometimes 

often 

Student Concepts of 

very bad for 

somewhat bad for somewhat good for 

very good for 

Democracy 

democracy 

democracy democray 

democray 

Student Concepts of 
Citizenship 

not important 

somewhat 

somewhat important 

unimportant 

very important 


a 

b c 

d 

Student Civic Knowledge & 
Interpretation Skill 


Multiple-choice 



The sample in this study was stratified into three categories, namely government schools in urban 
areas, government schools in mral areas and private schools. Eleven schools were selected from 
each stratum with the cluster size of 30 students for each school. As a result, the effective sample 
size of this study was 980 students. 


Sample 

The data used in this study were collected using survey instrument and involved 100 items and 
1030 ninth grade students in North Sulawesi, Indonesia. Because the responses obtained from 100 
cases were found to be unsatisfactory, with substantial missing data (more than 20 %), only 930 
cases were included in this analysis. 

Instrument Validation 

In order to validate the instrument developed, a pilot testing had been undertaken in June 2005 in 
which 200 hundred ninth grade students were involved. Principal Component Analysis (PCA) and 
Rasch Measurement Model Analysis (RMMA) led to the omission of several items that did not fit 
the Rasch Measurement Model. Infit mean square values in the range of 0.78 to 1.30 were used 
(Bond & Fox, 2001). Values less than 0.78 indicated significant overfit, and values greater that 
1.30 indicated significant underfit. Both underfitting and overfitting items were considered 
misfitting. However, in order to solve item shortage, five items with infit mean square (IMS) less 
than 0.78 were included in the final instrument. From this pilot testing and analysis, 17, 15, 34, 13 
and 21 items for Democratic Climate of Civic Education Classroom (DCCEC), Student 
Engagement in Civic Education Classroom (SECEC), Student Civic Knowledge and 
Interpretation Skill (SCKIS), Student Concepts of Democracy (SCD) and Student Concepts of 
Citizenship (SCC) scales respectively were selected to be used in the study. 

After handling the missing data using multiple imputation with NORM software (Darmawan, 
2002; Schafer, 1999), two analysis procedures were conducted, namely Confirmatory Factor 
Analysis using Mplus version 2.1 (Muthen & Muthen, 1998) and Rasch Measurement Model 
Analysis (RMMA) using Quest (Adams & Khoo, 1993). CFA was used to assess the 
multidimensionality of the scales and to compare their factorial models, whereas RMMA was 
used to recheck the unidimensionality of the scales (Bond & Fox, 2001). 
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In the Confirmatory Factor Analysis (CFA), factorial models (i.e. one factor and multiple factors 
including uncorrelated, hierarchical and nested models) were compared for each scale. This 
model comparison was carried out using several statistical fits, namely Chi-Square, Chi-Square 
divided by Degree of Freedom, Comparative Fit Index (CFI), Tucker- Lewis Index (TLI) and 
RSMEA. This analysis resulted in the formation of factors as observed variables (manifest 
variables) underlying five unobserved variables (latent variables). Out of four models compared 
for each scale, a hierarchical model showed superiority over other models except for Student 
Concepts of Democracy (SCD) scale where the one factor model was more adequate than others. 
This CFA results provided the factor structure shown in Table 3. 


Table 3. Factor structure in CFA result 


Latent Variables 


Manifest Variables 

1 . Democratic Climate 

1 . 

Clasroom participation (PRT) 

of Civic Education Classroom (DCCEC) 

2. 

Controversial issues (CONT) 


3. 

Reflective thinking (RFT) 


4. 

Relevant issues (RLV) 

2. Student Engagement 

1 . 

Behavioural engagement (BHV) 

in Civic Education Classroom (SECEC) 

2. 

Emotional engagement (EMT) 


3. 

Cognitive engagemetn (CGN) 

3. Student Civic Knowledge 

1 . 

Civic knowledge (KNW) 

& Interpretation Skill (SCKIS) 

2. 

Civic intrepretation Skill (INTRSKL) 

4. Student Concepts of Democracy (SCD) 

Unit}’ Concepts of Democracy (DEM) 

5. Student Concepts of Citizenship (SCC) 

1 . 

Understanding several ways to participate in civic life (PRTW) 


2. 

Understanding civic aspects to be respected (RESPT) 


3. 

Understading the nature of laws (L\Y) 


4. 

Student prudence in responding to different aspects of civic life (PRD) 


Items that had loadings less than 0.20 were excluded from the factors of each latent variable. In 
order to assure the unidimensionality of each construct or variable used, the selected items were 
then analysed using Rasch Measurement Model Analysis (RMMA) involving only fitting cases 
and the cases that completed all items or did not have zero or perfect scores. Items that initially 
fitted the Rasch model were selected to be the start values for estimating measures for each 
manifest variable in the second RMMA. In this second RMMA run, cases that were excluded 
initially were pulled back to be re-estimated using the anchoring method. The scores obtained 
from this estimation were then used as input data for further analysis in the study. 

Modelling 

Based on the literature reviewed in this study, a hypothesised model for the influence of 
democratic climate of civic education classroom on student engagement and on the civic learning 
outcomes was advanced (see Figure 1). It was hypothesised that democratic climate of a civic 
education classroom could influence student engagement in civic education and the civic 
education learning outcomes. 

Figure 1 represents the structure of the hypothesised model of the study so called Model A. This 
model consisted of 14 manifest variables (MVs) and 5 latent variables (LVs) that were produced 
through the previous Confirmatory Factor Analysis (CFA) and Rasch Measurement Model 
Analysis (RMMA). Democratic Climate of Civic Education Classroom (DCEC) scale, the 
independent variable (exogenous variable) in the model was formed as an inward or formative 
mode, whereas all other endogenous variables were constructed in the outward or reflective 
modes. It is worthy of note that Student Concepts of Democracy (SCD) had just one manifest 
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variable. The Partial Least Square (PLS) loading or weight for such a variable is always equal to 
unity regardless of its specification as inward or outward mode. 



Figure 1. Model A: Structure of the Hypothesised Model 

From Figure 1, it could be seen that the inner model structure of Model A formally comprised a 
recursive path model that in terms of direct effects had four restrictions. These restrictions were 
about direct effects of other variables on the variable of Student Concepts of Citizenship (SCD) 
which reflected outcomes, namely student understanding of ways to participate in civic life, 
aspects of civic life to be respected, nature of laws and student prudence in responding to 
different aspects of civic life. As shown in Figure 1, Democratic Climate of Civic Education 
Classroom (DCCEC), Student Engagement in Civic Education Classroom (SECEC), Student 
Civic Knowledge and Interpretation Skill (SCKIS) and Student Concepts of Democracy (SCD) 
were assumed to have direct effects on Student Concepts of Citizenship (SCC). Apart from these 
direct effects, DCCEC and SECEC were assumed to have indirect effects on SCC that operated 
through SCKIS and SCD. In addition, SCKIS had also an indirect effect on SCC that operated 
through SCD. 

This specification can be justified on the basis of an assumption that those students who are in a 
democratic civic education classroom will be inspired to engage and learn more actively. Through 
the democratic climate of the civic education classroom, they do not only have opportunities to 
learn actively but also to develop their social skills enabling them to participate effectively and 
responsibly in their civic education classrooms. Hence, a democratic climate does not only 
influence their cognitive development but also their social skill development. In addition, through 
engagement in the civic education classroom, they show their positive or negative attitudes 
towards civic education classrooms. Positive attitudes will result in a better approach to the Civic 
Education subject that will help them increase their civic knowledge and understanding. 
Furthermore, since a better approach means deeper exploration and more collaboration, this 
learning experience will also facilitate the development of their concepts about democracy and 
citizenship. 

The model of this study involved latent variables measured indirectly by different indicators. In 
order to handle this model, the data were analysed by employing Partial Least Square (PLS) 
method that was developed by Wold using statistical software called PLSPATH 3.01 (Sellin, 
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1990). This technique provides explicit estimation of latent variable scores by means of least 
square methods without requiring stringent distributional assumption (Sellin, 1995). 

For exploratory analysis purpose, Model A as a hypothesized model was compared with Model B. 
As a comparison of the inner model results could not be performed in this kind of analysis, a 
comparison was only made for the outer model statistics. The descriptive redundancies and the 
associated jackknife redundancies for each endogenous manifest variable (MV) along with the 
block total and model total were compared between Model A and Model B. This procedure gave 
the overall fit of the hypothesized model and the refined model (Sellin, 1995). This fit indicated 
the predictive power of each model. In addition, the residual path for each latent variable was also 
calculated using the following expression: 

r = Jl-R 2 

where r is a residual path and indicates the effects associated with unexplained variance in a 
certain latent variable and R 2 is the explained variance associated with latent variable. 
Furthermore, weights and loadings in the outer model and path coefficients, fit indices and model 
effects in the inner model obtained from data analysis were used in interpretation and testing of 
the model. 


RESULTS 

In order to assess how the measured variables might be linked to each other, how well the data 
fitted the model proposed and how the democratic climate of civic education classrooms might 
influence student engagement, civic knowledge and interpretation skill, concepts of democracy 
and citizenship, the initial model called ‘Model A’ was assessed and trimmed to produce a more 
adequate model in its predictive power and coherence called ‘Model B\ 

Tables 4 and 5 present the PLS results obtained based on the proposed model (i.e. Model A) and 
the refined model (i.e. Model B). 

Table 4. Weights and Loadings of Models A and B 




Model 


Variable 

A 


B 


(n=930) 

Weight 

Load 

Weight 

Load 

Democratic Climate 

Participation 

0.76 

0.85 

0.73 

0.88 

Controversial 

-0.36 

-0.20 

-0.27 

-0.04 

Reflective thinking 

0.48 

0.59 

0.52 

0.69 

Relevance 

-0.21 

-0.01 

-0.10 

0.16 

Student Classroom Engagement 

Behavioural Engagement 

0.41 

oo 

r- 

o 

0.42 

0.78 

Emotional Engagement 

0.48 

0.78 

0.46 

0.77 

Cognitive Engagement 

0.40 

0.77 

0.41 

0.78 

Civic Knowledge & 
Interpretation Skill 

Knowledge 

0.80 

0.92 

0.79 

0.92 

Intrepretation Skill 

0.40 

0.65 

0.41 

0.66 

Student Concepts of Democracy 

Concepts of Democracy 

1.00 

1.00 

1.00 

1.00 

Student Concepts of Citizenship 

Participation Ways 

0.32 

0.82 

0.32 

0.82 

Respect 

0.26 

0.77 

0.26 

0.77 

Law 

0.39 

0.84 

0.39 

0.84 

Prudence 

0.28 

0.78 

0.29 

0.78 
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Table 4 shows the PLS weights and loadings for each block. Table 5 displays the estimated direct 
inner model effects, the corresponding jackknife standard errors and the R-square and Q-square 
values for each inner model equation. All manifest variables were standardised prior to the PLS 
estimation. Therefore, the loadings and weights displayed in Table 4 are zero-order correlations 
between manifest variables (MVs) and their corresponding latent variables (LVs). In addition, the 
inner model coefficients in Table 5 are standardised path coefficients because the LVs are always 
standardised to unit variance (Sellin, 1995). 


Table 5. Estimated Direct Effects. Jackknife Standard Errors in Parenthesis 


Predicted 

Predictor 


Model 


LV (A '= 930 ) 


A 


B 

Student Engagement in Civic 

Democratic Climate of Civic Ed. 

0.43 

(0.03) 

0.44 

(0.03) 

Education Classroom (SECEC) 

Classroom (DCCEC) 






R 2 

0.18 


0.19 



Q 2 

0.18 


0.19 


Student Civic Knowledge & 

Democratic Climate of Civic Ed. 

0.14 

(0.04) 

0.18 

(0.03) 

Interpretation Skill (SCKIS) 

Classroom (DCCEC) 






Student Engagement in Civic Ed. 
Classroom (SECEC) 

0.07 

(0.03) 




R : 

0.03 


0.03 



Q 2 

0.03 


0.03 


Student Concepts of Democracy 

Democratic Climate of Civic Ed. 

0.09 

(0.05) 



(SCD) 

Classroom (DCCEC) 

Student Engagement in Civic Ed. 
Classroom (SECEC) 

0.11 

(0.04) 

0.15 

(0.03) 


Student Civic Knowledge & 
Interpretation Skill (SCKIS) 

0.24 

(0.03) 

0.26 

(0.03) 


R 1 

0.10 


0.10 



Q 2 

0.09 


0.09 


Student Concepts of Citizenship 

Democratic Climate of Civic Ed. 





(SCC) 

Classroom (DCCEC) 

0.22 

(0.03) 

0.23 

(0.03) 


Student Engagement in Civic Ed. 
Classroom (SECEC) 

0.25 

(0.03) 

0.24 

(0.04) 


Student Civic Knowledge & 
Interpretation Skill (SCKIS) 

0.16 

(0.03) 

0.15 

(0.03) 



0.16 

(0.03) 

0.17 

(0.03) 


Student Concepts of Democracy (SCD) 






R ' 

0.26 


0.26 



o 2 

0.25 


0.26 



Table 5 indicates that two direct effects in the model were found to be virtually zero because they 
did not meet the rule of thumb that weights and loadings must have twice the estimated standard 
errors (Keeves, Darmawan & Njora, 2003). This resulted into the deletion of the two 
corresponding paths in Model B. These paths were Student Engagement in Civic Education 
Classroom (SECEC) on Student Civic Knowledge and Interpretation Skill (SCKIS) and 
Democratic Climate of Civic Education Classroom (DCCEC) on Student Concepts of Democracy 
(SCD). 

With respect to the inner model, a comparison can be made between Model A and Model B if the 
weights are numerically the same. However, they are usually different because the weights 
depend on the specified inner model. Therefore, path coefficients, R 2 and Q in the inner model 
cannot be compared. What are compared in this analysis are the inner path estimates in terms of 
relative effect sizes (Sellin, 1995) as shown in Table 6. 

Table 5 shows that after the deletion of two paths (SECEC on SCKIS and DCCEC on SCD) from 
the model, there were changes in path coefficients. The effect of Democratic Climate of Civic 
Education Classroom (DCCEC) on Student Civic Knowledge and Interpretation Skill (SCKIS), 
the effect of Student Engagement in Civic Education Classroom (SECEC) on Student Concepts of 
Democracy (SCD) and the effect of Student Concepts of Democracy (SCD) on Student Concepts 
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of Citizenship (SCC) increased from 0.14 (0.04) to 0.18 (0.03), 0.11 (0.04) to 0.15 (0.03) and 
from 0.16 (0.03) to 0.17 (0.03) respectively. In addition to the direct effects, Table 6 shows 
indirect effects in both models. DCCEC had indirect effects on SCKIS through SECEC in Model 
A (0.03), on SCD thorough SCKIS in Model A and B (0.09) and through SECEC in Model B 
(0.11) and on SCC through SCKIS, SECEC and SCD in Model A (0.16) and through SCKIS in 
Model B (0.15). SECEC also had indirect effects on SCD through SCKIS in Model A (0.06) and 
on SCC through SCKIS and SCD in Model A (0.03) and through SCD in model B (0.02). 
Furthermore, SCKIS had an indirect effect on SCC through SCD in both models (0.38 and 0.04) 
respectively. 


Table 6. Direct and Indirect Effects of the Predictors on the Predicted 


Predictor Predicted 



Model 






A 



B 


LV LV 


Effects 



Effects 


(n=930) 

Direct 

Indirect 

Total 

Direct 

Indirect 

Total 

Democratic Student 







Climate Engagement 

0.43 

- 

0.43 

0.44 

- 

0.44 

Civic Knowledge & 
Interpretation Skill 
Concepts 

0.14 

0.03 

0.17 

0.18 

- 

0.18 

of Democracy 

0.09 

0.09 

0.1S 

- 

0.11 

0.11 

Concepts 
of Citizenship 

0.22 

0.16 

0.38 

0.23 

0.15 

0.38 

Student Civic Knowledge & 

Engagement Interpretation Skill 

0.07 


.07 




Concepts 
of Democracy 
Concepts 

0.11 

0.02 

0.13 

0.15 

- 

0.15 

of Citizenship 

0.25 

0.03 

0.28 

0.24 

0.02 

0.26 

Civic Knowledge & Concepts 
Interpretation Skill of Democracy 

0.24 


0.24 

0.26 


0.26 

Concepts 
of Citizenship 

0.16 

0.04 

0.20 

0.15 

0.04 

0.19 

Concepts of Concepts 







Democracy of Citizenship 

0.16 

- 

0.16 

0.17 

- 

0.17 


Even though the inner models could not be compared, it is possible to compare their outer model 
statistics using descriptive redundancies and the corresponding jackknife redundancies for each 
endogenous manifest variables along with block total and model total (Sellin, 1995). As shown in 
Table 7, the modifications had been made for Model A and Model B yielded only small changes 
in terms of predictive power because the difference in overall jackknife estimates between the two 
models was not significant, namely 0.112 to 0.115. However, this small difference to some extent 
indicates the superiority of Model B over Model A. Therefore, model modification done on 
Model A had increased its predictive power slightly (Sellin, 1995). 

The PLS results presented above suggest that Model B would be preferred to Model A. Apart 
from its parsimony, Model B was slightly more powerful to Model A in terms of prediction. 
However, this does not mean that Model B was so-called ‘true’ model because it was just one of 
many possible alternative models available. 

With respect to the direct effects, it could be seen from Table 5 that while Democratic Climate of 
Civic Education Classroom (DCCEC), Student Engagement in Civic Education Classroom 
(SECEC), Student Civic Knowledge and Interpretation Skill (SCKIS) and Student Concepts of 
Democracy (SCD) were found to be the most powerful predictors of Student Concepts of 
Citizenship (SCC), DCCEC turned out to be the most powerful predictor of Student Engagement 
in Civic Education Classroom (SECEC) and Student Civic Knowledge and Interpretation Skill 
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(SCKIS). On the other hand, SCKIS became the most powerful predictor of Student Concepts of 
Democracy (SCD). It should be noted that SCKIS was found to be weakly predicted by SECEC. 
On the contrary, SDC was weakly predicted by DCCEC. 

Table 7. Descriptive ( R j ) and Jackknife ( Rf) Estimates of Redundancies for Comparing 


Predictive Power of Model A and B 


V ariable 


Model 



A 

B 


02=930) 

R< 

R , 


R, 

Student Engagement 

Behavioural Engagement 

0.111 

0.108 

0.118 

0.115 

Emotional Engagement 

0.113 

0.110 

0.116 

0.113 

Cognitive Engagement 

0.110 

0.107 

0.117 

0.115 

Block Total 

0.334 

0.325 

0.351 

0.343 

Civic Knowledge & 
Interpretation Skill 

Knowledge 

0.028 

0.023 

0.027 

0.023 

Intrepretation Skill 

0.014 

0.011 

0.014 

0.012 

Block Total 

0.042 

0.034 

0.041 

0.035 

Student Concepts of Democracy 

Concepts of Democracy 

0.103 

0.093 

0.095 

0.089 

Block Total 

0.103 

0.093 

0.095 

0.089 

Student Concepts of Citizenship 

Participation Ways 

0.175 

0.169 

0.176 

0.170 

Respect 

0.156 

0.150 

0.157 

0.152 

Law 

0.183 

0.176 

0.184 

0.178 

Prudence 

0.160 

0.154 

0.161 

0.156 

Block Total 

0.674 

0.649 

0.678 

0.656 

Model total 

0.117 

0.112 

0.119 



With regard to the indirect effects, it should be seen from Table 6 that DCCEC had indirect 
effects on SCKIS, SCD and SCC. On the other hand, SECEC had an indirect effect on SCC. In 
addition, SCKIS had an indirect effect on SCC. Regardless of the effect types (i.e. direct or 
indirect), all predictors had significant effects on the Student Concepts of Citizenship (SCC) as an 
outcome in the model (see Figure 2). 



Figure 2. Model B: Structure of Refined Model from PLS Analysis 
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DISCUSSION 

PLS analysis was run to examine the predictive power of the proposed model so called Model A. 
The analysis showed that this model was slightly less coherent than the refined model so called 
Model B. This slight difference indicates that Model B has superiority on Model A in its 
parsimony and slightly more coherent in its theoretical adequacy. 

In general, all variables that were predicted to influence student concepts of citizenship were 
found to have significant path coefficients. This indicates that democratic climate of Civic 
Education classrooms, student engagement in Civic Education classrooms, student civic 
knowledge and interpretation skill and student concepts of democracy are four important factors 
that should be considered by teachers in classrooms in order to produce good citizens. 

In details, the democratic climate of Civic Education classrooms played an important role in 
fostering student civic knowledge and interpretation skill, student engagement and student 
concepts of citizenship in North Sulawesi. However, democratic climate was found not to 
contribute significantly to the level of student concepts of democracy. This implies the 
importance of developing a Civic Education classroom that is characterised with democratic 
climate where students are provided with chances to practice more reflective thinking. Such a 
classroom is more likely to facilitate students to obtain civic knowledge and interpretation skill, to 
make them engaged in Civic Education classrooms and to help them to develop concepts of 
citizenship properly. These results in general support the importance of social and cognitive 
environment of classrooms as it is suggested by previous researchers, such as Allodi (2002), 
Aikin as reported by Morgenstern and Keeves (1997) and Fredricks et al. (2004). The finding that 
democratic climate does not play statistically a significant role in developing student concepts of 
democracy in North Sulawesi is not easy to explain because other research (e.g. cross-national 
studies of Civic Education in 1975 and 2001 by Torney-Purta et al.) has suggested that 
democratic climate played important roles in developing the student understanding of democracy. 

In contrast to the democratic climate that was found to influence student civic knowledge and 
interpretation skill, student engagement was found to have a significant effect on student concepts 
of democracy, but not on student civic knowledge and interpretation skill. This result was 
unexpected because the review of literature indicated that student engagement could help students 
to develop their knowledge (e.g. Fredricks et al., 2002, 2004). Similar to this, democratic climate 
was strongly suggested to have an effect on student concepts of democracy (e.g. Torney-Purta, 
2001). This issue might be associated with the nature of student classroom engagement in North 
Sulawesi where engagement was shaped not to be compatible with the development of student 
civic knowledge. The engaging condition of classrooms created by classroom members, teachers 
and students, could not help students to practice cognitive engagement that was more likely to be 
associated with knowledge development (Anderson, 2000). In addition, democratic climate 
developed in Civic Education classrooms was more likely to be designed in a way that did not 
support the development of student concepts of democracy. 

It is understood from the review of literature that discussing controversial and relevant issues 
(Torney-Purta et al., 2001) are two important elements that are suggested to have significant 
effects on the student concepts of democracy in Civic Education classrooms. The presence of two 
manifest variables involving Controversial Issues and Relevance, that contributed negatively to 
the Democratic Climate of Civic Education Classroom (DCCEC) latent variable, on the one side, 
and the two other manifest variables involving Participation and Reflective Thinking that made 
positive contributions to it, on the other side, was an unexpected result. The previous research has 
suggested that raising controversial and relevant issues in Civic Education classrooms were the 
most important elements of the democratic climate scale (Kubow & Kinney, 2000; Tomey-Purta 
et al., 2001). Bearing on mind the possible limitation existing in the instrument battery, this might 
imply that controversial and relevant issues were discussed in Civic Education classrooms by 
using a method that did not support the shaping of democratic classrooms. For example, teachers 
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raised controversial issues in classrooms and put themselves as the ultimate resources of final 
opinions in front of their students. Such a practice might make students incapable to understand 
and to practice democracy in their classrooms. Another possibility was that teachers, in general, 
rarely raised controversial and relevant issues in classrooms so that students could not work out 
properly items addressing both sub-concepts in the instrument. 

It is also possible that what makes democratic climate and student engagement behave differently 
on student civic knowledge and interpretation skill, and on student concepts of democracy is the 
existing difference in their nature. Democratic climate represents both cognitive and attitudinal 
elements of learning condition simultaneously, whereas student engagement incorporates 
cognitive, behavioural and emotional engagement. It was shown in Table 4 that within democratic 
concept, participation and reflective thinking contributed more to the scale compared to relevant 
and controversial issues that had negative signs. It was also shown that within engagement 
concept, emotional engagement became the most dominant element in the scale compared to the 
behavioural and cognitive elements of sub-scales. As a result, it is assumed that the development 
of student civic knowledge and interpretation skill is more likely to be associated with the 
democratic climate compared to the student engagement due to its cognitive nature. In contrast, 
the development of student concepts of democracy is more likely to correspond to student 
engagement due to its attitudinal nature. Furthermore, the development of student concepts of 
citizenship was found to be affected by both democratic climate and engagement because 
concepts of citizenship are mix of emotional, attitudinal and cognitive representations of 
knowledge. This explanation is based on the notion of multiple intelligences (Anderson, 2000) 
that leads to the assumption that each kind of specific knowledge requires different conditions to 
be nurtured. 

Student civic knowledge and interpretation skill that were found to have significant effects both 
on student concepts of democracy and citizenship indicated that in order to produce a democratic 
and responsible citizens, students should be provided with a better understanding of civic 
knowledge and skill. Previous cross-national studies led by Torney-Purta in 1975, 1999 and 2001 
strongly suggest that civic knowledge and interpretation skill are important to help students to 
have a better understanding of democracy and citizenship. 

Finally, student concepts of democracy that were also found to have a significant effect on student 
concepts of citizenship indicated that in order to produce good citizens, to some extent, a good 
understanding of democracy was required. 

CONCLUSIONS 

The refined model so called Model B in this study was proved to be more parsimonious and 
slightly more coherent in its theoretical adequacy. In addition, all variables that were predicted to 
influence student concepts of citizenship, namely democratic climate of Civic Education 
classrooms, student engagement in Civic Education classrooms, student civic knowledge and 
interpretation skill and student concepts of democracy were found to have significant path 
coefficients in the model tested. Democratic climate of Civic Education classrooms played an 
important role in fostering student civic knowledge and interpretation skill, student engagement 
and student concepts of citizenship in North Sulawesi. However, democratic climate was found 
not to contribute significantly to the level of student concepts of democracy. 

Unlike democratic climate, student engagement was found to have a significant effect on student 
concepts of democracy, but not on student civic knowledge and interpretation skill. In addition, 
student civic knowledge and interpretation skill were found to have significant effects both on 
student concepts of democracy and citizenship. Furthermore, student concepts of democracy were 
also found to have a significant effect on student concepts of citizenship. Finally, this finding 
should initially be tested and replicated in further studies before acceptance. If the evidence 
supports the finding emerging from this study, then greater thought should be given to the making 
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of a democratic and engaging classroom climate and to what is involved in building such a 
classroom environment. 
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